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Abstract A Bayesian estimation of a GARCH model is performed for US Dol- 
lar/Japanese Yen exchange rate by the MetropoUs-Hastings algorithm with a pro- 
posal density given by the adaptive construction scheme. In the adaptive construc- 
tion scheme the proposal density is assumed to take a form of a multivariate Stu- 
dent's t-distribution and its parameters are evaluated by using the sampled data and 
updated adaptively during Markov Chain Monte Carlo simulations. We find that the 
autocorrelation times between the data sampled by the adaptive construction scheme 
are considerably reduced. We conclude that the adaptive construction scheme works 
efficiently for the Bayesian inference of the GARCH model. 
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1 Introduction 

In finance volatility of asset returns plays an important role to manage financial risk. 
To forecast volatility, various empirical models which mimic the properties of the 
volatility have been proposed. Engle|r| proposed Autoregressive Conditional Het- 
eroskedasticity (ARCH) model where the present volatility is assumed to depend on 
the squares of past observations. Later Bollerslev0 advocated Generalized ARCH 
(GARCH) model which is an extension of the ARCH model and includes addi- 
tional past volatility terms to the present volatility estimate. It is known that the 
volatility of the financial assets exhibits clustering in the financial time series. The 
GARCH model can captures this property. Furthermore the return distribution gen- 
erated from the GARCH process shows a fat-tailed distribution which is also seen in 
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the real financial markets. There also exists extension of the GARCH model which 
incorporates the asymmetric property of the volatility ||3] ID |5]|6l. 

A preferred algorithm to infer GARCH model parameters is the Maximum Like- 
lihood (ML) method which estimates the parameters by maximaizing the corre- 
sponding likelihood function of the GARCH model. In this algorithm there is a 
practical difficulty in the maximization procedure when the output results are sensi- 
tive to starting values. 

By the recent computer development the Bayesian inference by Markov chain 
Monte Carlo (MCMC) methods, which is an alternative approach to estimate 
GARCH parameters, has become popular There exist a variety of methods pro- 
posed to implement the MCMC scheme 171 - lfT2l . In a recent survev lfTTl it is shown 
that Acceptance-Rejection/Metropolis-Hastings (AR/MH) algorithm works better 
than other algorithms. In the AR/MH algorithm the proposal density is assumed to 
be a multivariate Student's t-distribution and the parameters to specify the distribu- 
tion are estimated by the ML technique. Recently a new method to estimate those 
parameters without relying on the ML technique was proposed (131. In the method 
the parameters are determined by an MCMC simulation. During the MCMC simu- 
lation, the parameters are updated adaptively using the data sampled by the MCMC 
method itself. We call this method "adaptive construction scheme". The adaptive 
construction scheme was tested for artificial GARCH data and it is shown that the 
adaptive construction scheme can significantly reduce the correlation between sam- 
pled data||T3l|. In this study we apply the adaptive construction scheme to real finan- 
cial data, US Dollar/Japanese Yen exchange rate and examine the efficiency of the 
adaptive construction scheme. 



2 GARCH Model 

The GARCH(p,q) model by BollerslevQ is given by 

yt = o,£,, (1) 

a}^co+Y. (^y^-i + L Ao-r-,- , (2) 

1=1 1=1 

where the GARCH parameters are restricted to o > 0, «; > and j3, > to ensure a 
positive volatility, and the stationary condition ^?^[ OLi + Y!i= \ Pi < 1 is also required. 
Et is an independent normal error ~ A^(0, 1). 

In this study we focus on GARCH(1,1) model where the volatiUty is given by 

= (0 + ayj_i+Pa^_i. (3) 

The likelihood function of the GARCH model is given by 
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L(y|0)=n;Li-=L=exp(-^). (4) 



3 Bayesian inference 

Using Bayes' rule the posterior density %{Q\y) with n observations denoted by y = 
(yi,}'2, ■■■,}'«) is given by 

n{B\y)^L{y\Q)n{B), (5) 

where L(y\Q) is the likeUhood function. n{Q) is the prior density which we have 
to specify depending on 0. In this study we assume that the prior density 7r(0) is 
constant. 

With we infer as expectation values of Q. The expectation values are 

given by 

{9) = ^j97zie\y)d9, (6) 

where Z = J n{0\y)dO is the normalization constant. Hereafter we omit Z since this 
factor is irrelevant to MCMC estimations. 

The MCMC technique gives a method to estimate eq.® numerically. The basic 
procedure of the MCMC method is as follows. First we sample 9 drawn from a 
probability distribution n{9\y). Sampling is done by a technique which produces a 
Markov chain. After samphng some data, we evaluate the expectation value as an 
average value over the sampled data 9^'\ 




where k is the number of the sampled data. The statistical error for k independent 
data is proportional to When the sampled data are correlated the statistical error 

will be proportional to where t is the autocorrelation time between the sampled 
data. The autocorrelation time depends on the MCMC method we employ. Thus it 
is desirable to take an MCMC method which can generate data with a small T. 



4 Metropolis-Hastings algorithm 

The Metropolis-Hastings (MH) algorithm lfT4l is an MCMC simulation method 
which generates draws from any probability density. The MH algorithm is an exten- 
sion of the original Metropolis algorithm lfTSl . Let us consider a probability distribu- 
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tion P{x) from which we would like to sample data x. The MH algorithm consists 
of the following steps. 

( 1 ) First we set an initial value xq and / = 1 . 

(2) Then we generate a new value x, from a certain probability distribution 
g{xi\xi-i) which we call proposal density. 

(3) We accept the candidate jc,- with a probability of PMH{xi-i,Xi) where 



PMHixi-i,Xi) ^min 



P{Xi) g{xi\xi^]) 



' P{Xi-l) g{Xi-l\xi) 



(8) 



When X, is rejected we keep x,_ i , i.e. x,- = x,_ i . 
(4) Go back to (2) with an increment of ; = ; + 1 . 

For a symmetric proposal density ^(x,|x,_i) = ^(x,_i |x,), eq.® reduces to the 
Metropolis accept probability: 



PMetro{xi-i,Xi) = min 



P(x,-) 



P{xi-i) 



(9) 



5 Adaptive construction scheme 



Since the proposal density g (x, |x,_ i ) is dependent of the previous value x,_ i , usually 
the sampled data are correlated. One may use an independent proposal density §(x,) 
which does not depend on the previous value. Although in this case we can generate 
independent candidates x,, it is important to choose the one close enough to the 
posterior density, in order to make the acceptance high enough. 

The posterior density of GARCH parameters often resembles to a Gaussian-like 
shape. Thus one may choose a density similar to a Gaussian distribution as the 
proposal density. Following ifTOl [TTl . in order to cover the tails of the posterior 
density we use a (p-dimensional) multivariate Student's t-distribution given by 



r((v+p)/2)/r(v/2) 



detZi/2(v7r)p/2 
where 9 and M are column vectors. 



{B - M)' L-\e - M) 



-(v+p)/2 



(10) 
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(11) 



andM, = £(0,). E is the covariance matrix defined as 

vE 



v-2 



= £[(0-M)(0-M)']. 



(12) 
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For later use we also define a matrix V as 



V = E[{e-M){e-M)']. 



(13) 



V is a parameter to tune the shape of Student's t-distribution. When v — > oo the 
Student's t-distribution goes to a Gaussian distribution. In this study we take v = 10. 

There are three parameters to be inferred for the GARCH(1,1) model. Therefore 
in this case p = 3 and 9 ^ {01,62,93) ~ {a, P ,(o), and Z is a 3 x 3 matrix. The val- 
ues of Z and M are not known a priori. We determine these unknown parameters M 
and Z through MCMC simulations. First we make a short run by the Metropolis al- 
gorithm and accumulate some data. Then we estimate M and E. Note that there is no 
need to estimate M and E accurately. Second we perform an MH simulation with the 
proposal density of eq.lfTOli with the estimated M and E. After accumulating more 
data, we recalculate M and E, and update M and E of eq.lfTOli. By doing this, we 
adaptively change the shape of eq.dTOli to fit the posterior density more accurately. 
We call eg. ([Toll with the estimated M and E "adaptive proposal density". 

The random number generation for the multivariate Student's t-distribution can 
be done easily as follows. First we decompose the symmetric covariance matrix E 
by the Cholesky decomposition as E = LL' . Then substituting this result to eq.dTOli 
we obtain 



follows A^(0,/) and w is taken from the chi-square distribution v degrees of freedom 
Xy- Finally we obtain the random number 9 hy 9 = LX +M. 



6 Empirical analysis 

We make an empirical analysis based on daily data of the exchange rates for US 
Dollar and Japanese Yen. The sampling period of the exchange rates is 4 January 
1999 to 29 December 2006, which gives 2006 observations. The exchange rates pi 
are transformed to r, = 100[ln(/7,//:',_i) — ,f] where i stands for the average value of 
HPi/Pi-i)- 

Our implementation of the adaptive construction scheme is as follows. First we 
make a short run by the Metropolis algorithm. We discard the first 3000 data as 
burn-in process. Then we accumulate 1000 data to estimate M and E. The estimated 
M and E are substituted to g{9) of eq.(fTO]i. The shape parameter v is set to 10. 
We re-start a run by the MH algorithm with the proposal density g{9). Every 1000 
update we re-calculate M and E using all accumulated data and update g{9) for the 
next run. We accumulate 100000 data for analysis. 

We also make a Metropolis simulation and accumulate 100000 data for analysis. 
The Metropohs algorithm in this study is implemented as follows. We draw a new 0' 




(14) 




6 



Tetsuya Takaishi 



Table 1 Results of parameters. 

a p CO 

Adaptive construction 0.03151 0.9403 0.01104 

standard deviation 0.0078 0.017 0.0047 

statistical error 0.00004 0.0001 0.00003 

2^ 2.8±0.3 3.8±0.4 4.1±0.5 

Metropolis 0.0318 0.9391 0.0114 

standard deviation 0.0079 0.018 0.005 

statistical error 0.0005 0.0014 0.0004 

2t„„ 400 ±60 650 ±100 620 ±80 



by adding a small random value 50 to the present value 9 ~ (Oi, 62,03) = {a, j3 ,co): 

e'j = ej + 8ej, (i5) 

where 5Gj = d{r — 0.5). r is a uniform random number in [0, 1] and d is a constant 
to tune the Metropolis acceptance. We choose d so that the acceptance becomes 
greater than 50%. 
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Fig. 1 Monte Carlo time histories of a sampled by the adaptive construction scheme (left) and 
the Metropolis algorithm(right). 



Fig. 1 compares the Monte Carlo time history of a sampled by the adaptive 
construction scheme with that by the Metropolis algorithm. It is clearly seen that the 
data a produced by the Metropolis algorithm are very correlated. On the other hand 
the sampled data by the adaptive construction scheme seem to be well de-correlated. 
For other parameters j3 and (O we also see the similar behavior 

In order to see correlations between sampled data, we measure the autocorrela- 
tion function (ACF) defined as 

ACF{t)= ^^^-'^ ^ '\ (16) 



Bayesian estimation of GARCH model with an adaptive proposal density 7 

where {x) and are the average value and the variance of certain successive data 
X respectively. 
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Fig. 2 Autocorrelation functions of a for the adaptive construction scheme (left) and the Metropo- 
lis algorithm (right). 



Fig. 2 shows the ACF for the adaptive construction scheme and the Metropo- 
lis algorithm. The ACF of the the adaptive construction scheme decreases quickly 
as Monte Carlo time t increases. On the other hand the ACF of the Metropolis 
algorithm decreases very slowly which indicates that the correlation between the 
sampled data is very large. 

We estimate the autocorrelation time by the integrated autocorrelation time T,„r. 
To calculate T,„f we define T,„r(r) as 

1 ^ 

%n{T) = - + Y,ACF{i). (17) 

^ i=i 

Tint is given by T;„((r = °o). In practice, however, it is impossible to sum up ACF{t) 
to r = oo. Since typically Tj„,{T) increases with T and reaches a plateau we estimate 
Tint at this plateau. Fig. 3 illustrates Tmt{T) of a sampled by the adaptive construc- 
tion scheme. T,„r {T) increases with T and reaches a plateau around T > 20. 

Results of T,„f are summarized in Table 1 . The values of T„„ from the Metropolis 
simulations are very large, typically several hundreds. On the other hand t„h from 
the adaptive construction scheme are very small, 2t,„i ^ 2 — f]. This results in a 
factor of 10 reduction in terms of the statistical error This reduction property is 
confirmed by the statistical errors of the sampled data (See Table 1). Thus it is 
concluded that the adaptive construction scheme is effectively working for reducing 
the correlations between the sampled data. 

Fig. 4 shows the convergence property of the matrix V. The matrix elements 
Vij are defined by V = E[{0 -M)(0-M)'] with = 02,03) = («,j3,to). For 



' 2t,„, is called an inefficiency factor. 
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Fig. 3 The integrated autocorrelation time 'r„„(r) of a sampled by the adaptive construction 
scheme. 
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Fig. 4 The matrix elements of the symmetric covariance matrix V. Diagonal elements (left) and 
off-diagonal elements (right). 



instance V12 = Vaji- All elements of V converge quickly to certain values as the 
simulations are proceeded. 

Fig. 5 shows the acceptance at the MH algorithm with the adaptive proposal 
density of eq.lfTOli. Each acceptance is calculated every 1000 updates and the cal- 
culation of the acceptance is based on the latest 1000 data. At the first stage of the 
simulation the acceptance is low. This is because at this stage M and E are not cal- 
culated accurately yet. However the acceptances increase quickly as the simulations 
are proceeded and reaches a plateau where the acceptance is more than 70%. 
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Fig. 5 Acceptance at MH step with the adaptive proposal density. 

7 Summary 

We proposed the adaptive construction scheme to construct a proposal density for 
the MH algorithm of the GARCH(1,1) model. The construction of the proposal den- 
sity is performed using the data generated by MCMC methods. During the MCMC 
simulations the proposal density is updated adaptively. In this study we applied the 
adaptive construction scheme for the Bayesian inference of the GARCH(1,1) model 
by using US Dollar/Japanese Yen exchange rate. The numerical results show that 
the adaptive construction scheme significantly reduces the correlations between the 
sampled data. The autocorrelation time of the adaptive construction method is cal- 
culated to be 2t„„ ~ 2 — 3, which is comparable to that of the AR/MH method lfTDl . 
It is concluded that the adaptive construction scheme is an efficient method for the 
Bayesian inference of the GARCH(1,1) model. The adaptive construction scheme 
is not Umited to the GARCH(1,1) model and can be applied for other GARCH-type 
models. 
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